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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH (S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)IE Responsive to communication(s) filed on 29 March 2004 . 
2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1^3 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1^3 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) S The drawing(s) filed on 29 March 2004 is/are: a)D accepted or b)S objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Drawings 

1 . Color photographs and color drawings are not accepted unless a petition filed 
under 37 CFR 1 .84(a)(2) is granted. Any such petition must be accompanied by the 
appropriate fee set forth in 37 CFR 1 .1 7(h), three sets of color drawings or color 
photographs, as appropriate, and, unless already present, an amendment to include the 
following language as the first paragraph of the brief description of the drawings section 
of the specification: 

The patent or application file contains at least one drawing executed in color. Copies of this patent or 
patent application publication with color drawing(s) will be provided by the Office upon request and 
payment of the necessary fee. 

Color photographs will be accepted if the conditions for accepting color drawings 
and black and white photographs have been satisfied. See 37 CFR 1.84(b)(2). 

Claim Rejections - 35 USC § 103 

1. Claims 1-3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Malik 
et al. (NPL Document, "Recovering Photometric Properties of Architectural Scenes 
From Photographs", herein referred to as "Malik") in view of Marchant et al. (NPL 
Document "Shadow-invariant classification for scenes illuminated by daylight", herein 
referred to as "Marchant"). 

As per claims 1 , 2, and 3, Malik teaches the claimed "obtaining a 3D geometrical 
model expressing a 3D form of an object by using geometrical information" by teaching 
of "a geometric model of the architecture" (3rd paragraph under section 1). 
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Malik teaches the claimed "obtaining geographical information including latitude 
and longitude indicating a position of said object and orientation of the object" by 
teaching of "The solar position(altitude and azimuth) can be obtained from formula given 
in the appendix of [18], provided that the latitude and longitude of the site on the earth's 
surface, and the time and date are known" (1st paragraph under section 3.1). 

Malik teaches the claimed "obtaining image data of said object shot with the sun 
as a light source and correspondence information indicating correspondence between a 
scene expressed by said image data and said 3D geometrical model as to their 
positions and forms" by teaching of "we take several photographs-of the sun, the sky, 
the architecture, and the environment surrounding the architecture" (3rd paragraph 
under section 1) and by teaching of "We take multiple photographs at different times 
and viewing directions( Figure 5), such as grazing angles, directions close to mirror 
angles of the solar positions and other directions, to sample the radiance distribution 
from the architecture" (1st paragraph above section 4.2.1). 

Malik teaches the claimed "obtaining shooting information including information 
on a shooting time and shooting situation of said image data" by teaching of 
"Photographs are taken for the sun, the sky, the landscape, as well as the architecture 
at a few different times of day to collect enough data for recovering the various lighting 
and reflectance models" (abstract). 

Malik teaches the claimed "placing said 3D geometrical model in a 
predetermined local coordinate system based on said geographical information" by 
teaching of "we have the following reflection model expressed in a local coordinate 
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system of each triangular patch" where the 3D geometrical model uses one or more 
reflection models. 

Malik teaches the claimed "calculating a light source direction in said local 
coordinate system by using said geographic information and said shooting time" by 
teaching of "the sun as a directional light source" (3rd paragraph under section 5.2) and 
by teaching of "the different positions of the light source are generated by the movement 
of the sun during the day" (3rd paragraph under section 5.2). 

Malik teaches the claimed "detecting a shadow region cast on a surface of said 
3D geometrical model by a beam in said light source direction by using said light source 
direction so as to identify the shadow region of said image data based on said 
correspondence information" by teaching of "When large shadows are inavoidable, we 
can interactively label the shadow boundaries to separate sunlit regions from shadowed 
ones" (1st paragraph above section 4.2) where labeling boundaries would require a 
least some knowledge of where they are located (through detection). 

Malik teaches the claimed "using a predetermined reflection model to estimate 
effects of shadings caused to said 3D geometrical model by the beam in said light 
source direction" by teaching of "The reflectance models are for the surfaces of the 
architecture" (abstract) and by teaching of "we proposed a method to extend image- 
based modeling and rendering techniques to deal with producing renderings under 
novel lighting conditions" (1st paragraph under section 7). 

Malik teaches the claimed "determining a parameter of the reflection model 
suited to said estimated shading" by teaching of "to solve a series of optimization 
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problems to find the parameters of appropriate lighting and reflectance models" 
(abstract). 

Malik teaches the claimed "fit ... to said 3D geometrical model" by teaching of "fit 
empirical or physics-based models to measured data and then using the obtained 
model into illumination calculation" (1st paragraph under section 4) 

Malik does not explicitly teach the claimed "performing calculation for removing 
the effects of the shadows and shadings by using said parameter from pixel values 
samples from said image data based on said correspondence information". Marchant 
teaches the claimed limitation by teaching of "What we do hope to remove are large 
shadows caused, for example, by vehicles, equipment, and people near the images" 
(1st full paragraph in 2nd col on pg. 1959) and by teaching of 'This means that effects 
due to shading will be removed" (last paragraph in 2nd col on pg. 1954). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine Malik and Marchant. One advantage to the combination is 
provided by Marchant, which teaches of "Shadows are often seen as a nuisance in 
images and are dealt with by use of image processing techniques" (last paragraph in 
2nd col on pg. 1952) and by teaching of the need to "achieve color constancy in the 
presence of shadows" (1st paragraph on pg. 1953). Malik teaches the desire for the 
functionality to remove shadows by teaching of "we proposed a method to extend 
image-based modeling and rendering techniques to deal with producing renderings 
under novel lighting conditions" (1st paragraph under section 7) where novel lighting 
conditions may be achieved easier by removing shadows using Marchant. 
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Conclusion 

2. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: Please see form 892. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Daniel F. Hajnik whose telephone number is (571 ) 272- 
7642. The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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